SMOOTHED ANALYSIS OF BALANCING NETWORKS* 



TOBIAS FRIEDRICHS THOMAS SAUERWALD^ AND DAN VILENCHIK^ 

Abstract. In a balancing network each processor has an initial col- 
lection of unit-size jobs (tokens) and in each round, pairs of processors 
connected by balancers split their load as evenly as possible. An ex- 
cess token (if any) is placed according to some predefined rule. As it 
turns out, this rule crucially affects the performance of the network. 
In this work we propose a model that studies this effect. We suggest 
a model bridging the uniformly-random assignment rule, and the arbi- 
trary one (in the spirit of smoothed-analysis). We start with an arbi- 
trary assignment of balancer directions and then flip each assignment 
with probability a independently. For a large class of balancing net- 
works our result implies that after ©(logn) rounds the discrepancy is 
©((1/2 — a) logn + log log n) with high probability. This matches and 
generalizes known upper bounds for a = and a = 1/2. We also show 
that a natural network matches the upper bound for any a. 



1. Introduction 

In this work we are concerned with two topics whose name contains the 
word "smooth", but in totally different meaning. The first is balancing 
( smoothing ) networks, the second is smoothed analysis. Let us start by intro- 
ducing these two topics, and then introduce our contribution - interrelating 
the two. 

1.1. Balancing (smoothing) networks. In the standard abstraction of 
smoothing (balancing) networks [2], processors are modeled as the vertices 
of a graph and connection between them as edges. Each process has an 
initial collection of unit-size jobs (which we call tokens). Tokens are routed 
through the network by transmitting tokens along the edges according to 
some local rule. We measure the quality of such a balancing procedure by 
the maximum difference between the number of tokens at any two vertices 
at the end. 
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Figure 1. The network CCCie- 



The local scheme of communication we study is a balancer gate: the 
number of tokens is split as evenly as possible between the communicat- 
ing vertices with the excess token (if such remains) routed to the vertex 
towards which the balancer points. More formally, the balancing network 
consists of n vertices vi,V2, ■ ■ ■ ,Vn, and m matchings (either perfect or not) 
Mi,M2, ■ ■ ■ ,Mm- We associate with every matching edge a balancer gate 
(that is, we think of the edges as directed edges). At the beginning of 
the first iteration, Xj tokens are placed in vertex vj, and at every itera- 
tion r = 1, . . . ,m, the vertices of the network perform a balancing opera- 
tion according to the matching (that is, vertices Vi and vj interact if 

{Vi,Vj) G Mr). 

One motivation for considering smoothing networks comes from the 
server-client world. Each token represents a client request for some ser- 
vice; the service is provided by the servers residing at the vertices. Routing 
tokens through the network must ensure that all servers receive approxi- 
mately the same number of tokens, no matter how unbalanced the initial 
number of tokens is (cf . [2] ) . More generally, smoothing networks are attrac- 
tive for multiprocessor coordination and load balancing applications where 
low-contention is a requirement; these include producers- consumers flT] and 
distributed numerical computations |3|]. Together with counting networks, 
smoothing networks have been studied quite extensively since introduced in 
the seminal paper of Aspnes et al. [1] . 



Herlihy and Tirthapura |12| . Il3l | initiated the study of the CCC network 
(cube-connected-cycles, see Figure [I]) as a smoothing networl£|. For the 
special case of the CCC, sticking to previous conventions, we adopt a "topo- 
graphical" view of the network, thus calling the vertices wires, and looking 
at the left-most side of the network as the "input" and the right-most as 



"'^ Actually, they considered the so-called block network. However, it was observed in [l^ 
that the block network is isomorphic to the CCC-network and therefore we will stick to 
the latter in the following. 
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the "output". In the CCC, two wh'es at layer £ are connected by a balancer 
if the respective bit strings of the wires differ exactly in bit I. The CCC is a 
canonical network in the sense that it has the smallest possible depth (num- 
ber of rounds) of logn as a smaller depth cannot ensure any discrepancy 
independent of the initial one. Moreover, it has been used in more advanced 
constructions such as the 'periodic (counting) network [1, 0|. 

As it turns out, the initial setting of the balancers' directions is crucial. 
Two popular options are an arbitrary orientation or a uniformly random one. 
A maximal discrepancy of log n was established for the CCC^ for an arbitrary 
initial orientation [l^. For a random initial orientation of the CCC,i, 
show a discrepancy of at most 2.36\/log n' for the CCC^ (this holds wh'^ 
over the random initialization), which was improved by Mavronicolas and 
Sauerwald [H] to log logn + 0(1) (and a matching lower bound). 

Results for more general networks have been derived in Rabani, Sinclair, 
and Wanka for arbitrary orientations. For expander graphs, they show 
an C?(log n)-discrepancy after C'(log(Kn)) rounds when K is the discrepancy 
of the initial load vector. This was recently strengthened assuming the 
orientations are set randomly and in addition the matchings themselves are 
chosen randomly Specifically, for expander graphs one can achieve within 
the same number of rounds a discrepancy of only ©(log logn). 

1.2. Smoothed analysis. Let us now turn to the second meaning of 
"smoothed". Smoothed analysis comes to bridge between the random in- 
stance, which typically has a very specific "unrealistic" structure, and the 
completely arbitrary instance, which in many cases reflects just the worst 
case scenario, and is thus over-pessimistic in general. In the smoothed anal- 
ysis paradigm, first an adversary generates an input instance, then this 
instance is randomly perturbed. 



The smoothed analysis paradigm was introduced by lSpielman and Teng 
in 2001 [3] to help explain why the simplex algorithm for linear program- 
ming works well in practice but not in (worst-case) theory. They considered 
instances formed by taking an arbitrary constraint matrix and perturbing it 
by adding independent Gaussian noise with variance e to each entry. They 
showed that, in this case, the shadow- vertex pivot rule succeeds in expected 
polynomial time. Independently, Bohman, Frieze, and Martin 0] studied 
the issue of Hamiltonicity in a dense graph when random edges are added. 
In the context of graph optimization problems we can also mention [8, 14 1, 
in the context of A;-SAT fa, and in various other problems [H. [isl. [l8. 2C| 



Our work joins this long series of papers studying perturbed instances in a 
variety of problems. Specifically in our setting we study the following ques- 
tion: what if the balancers were not set completely adversarially but also 
not in a completely random fashion. Besides the mathematical and analyt- 
ical challenge that such a problem poses, in real network applications one 
may not always assume that the random source is unbiased, or in some cases 



^Writing w]vp we mean with probability tending to 1 as n goes to infinity. 
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one will not be able to quantitatively measure the amount of randomness 
involved in the network generation. Still it is desirable to have an estimate 
of the typical behavior of the network. Although we do not claim that our 
smoothed-analysis model captures all possible behaviors, it does give a rig- 
orous and tight characterization of the tradeoff between the quality of load 
balancing and the randomness involved in setting the balancers' directions, 
under rather natural probabilistic assumptions. 

As far as we know, no smoothed analysis framework was suggested to a 
networking related problem. Formally, we suggest the following framework. 

1.3. The Model. We define both the smoothed-analysis aspect of the 
model, and the load-balancing one. For the load balancing part, our model 
is similar to (and, as we will shortly explain, a generalization of) the peri- 
odic balancing circuits studied in p^]. We think of the balancing network in 
terms of an n-vertex graph. The processors in the network are the vertices 
of the graph, and balancers connecting processors are the (directed) edges 
of the graph. 

Before we proceed, since what follows is somewhat heavy on notation 
and indices, it will be helpful for the reader to bear in mind the following 
legend: we use superscripts (in round brackets) to denote a time stamp, 
and subscripts to denote an index. In subscripts, we use the vertices of 
the graph as indices (thus assuming some ordering of the vertex set). For 
example, A^*^^ stands for the (u, f )-entry in matrix A^*) , which corresponds 
to time/round i. 

Let M^^^ , • • • , M^^^ be an arbitrary sequence of T (not necessarily perfect) 
matchings. With each matching M^*) we associate a matrix P^*) with P^*^ = 
1/2 if u and v are matched in M^, p'H = 1/2 if u is matched in M(*\ 

Pmm = 1 if u is not matched in M^*), and Pul = otherwise. 

In round i, every two vertices matched in M^*^ perform a balancing op- 
eration. That is, the sum of the number of tokens in both vertices is split 
evenly between the two, with the remaining token (if exists) directed to the 
vertex pointed by the matching edge. 

Remark 1.1. In periodic balancing networks (see fP)i ] for example) 
an ordered set of d (usually perfect) matchings is fixed. Every round of 
balancing is a successive application of the d matchings. Our model is a 
(slight) generalization of the latter. 

Let us now turn to the smoothed-analysis part. Given a balancing net- 
work consisting of a set T of directed matchings, an a-perturbation of the 
network is a flip of direction for every edge with probability a independently 
of all other edges. Setting a = gives the completely "adversarial model", 
and a = 1/2 is the uniform random case. 

Remark 1.2. For our results, it suffices to consider a G [0, 1/2]. The case 
a > 1/2 can be reduced to the case a ^ 1/2 by flipping the initial orientation 
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of all balancers and taking 1 — a instead of a. It is easy to see that both 
distributions are identical. 

1.4. Our Contribution. For a load vector x, its discrepancy is defined to 
be maxu^v — We use to denote tlie unit vector whose all entries 
are except the u*^. For a matrix A, X{A) stands for the second largest 
eigenvalue of A (in absolute value). Unless stated otherwise, ||2;|| stands for 
the ^2-iiorm of the vector z. In the following, we will assume an ordering 
of the vertices from 1 to n. When we write (u, v) G E, we refer to the case 
where u and v are connected by an undirected edge and u < v. 

Theorem 1.1. Let G be a balancing network with matching s 
M(i),...,M(^). For any two time stamps ti,t2 satisfying ti < t2 ^ T, and 
any input vector with initial discrepancy K, the discrepancy at time step t2 
in a-perturbed G is whp at most 

{t2 - ti) + 3 (i - a) ti + Ai + A2, 

where 

Ai = max^logn Y!U E[.:.]eM« ((^n - e.)^ 

A2 = A (nti P^'^) 

Before we proceed let us motivate the result stated in Theorem ll.il There 
are two factors that affect the discrepancy: the fact that tokens are indi- 
visible (and therefore the balancing operation may not be "perfect"), and 
how many balancing rounds are there. On the one hand, the more rounds 
there are the more balancing operations are carried, and the smoother the 
output is. On the other hand, the longer the process runs, its susceptibility 
to rounding errors and arbitrary placement of excess tokens increases. This 
is however only a seemingly tension, as indeed the more rounds there are, 
the smoother the output is. Nevertheless, in the analysis (at least as we 
carry it), this tension plays part. Specifically, optimizing over these two 
contesting tendencies is reflected in the choice of ti and t2- ^2 is the contri- 
bution resulting from the number of balancing rounds being bounded, and 
Ai, along with the first two terms, account for the indivisibly of the tokens. 
In the cases that will interest us, ti,t2 will be chosen so that Ai, A2 will be 
low-order terms compared to the first two terms. 

Our Theorem 1 1 . 1 1 also implies the following results: 

• For the aforementioned periodic setting Theorem 11.11 implies the 
following: after O {\og{Kn) /v) rounds (i^ = (1 — A(Q))~-'^), Q is 
the so-called round matrix which corresponds to one period, K the 
initial discrepancy) the discrepancy is whp at most 

^ ^ d \og{Kn) f\_^ dloglogn 
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Setting a = (and assuming K is polynomial in n) we get the 
result of and for a = 1/2 we get the result of i- (The 

restriction on K being polynomial can be lifted but at the price 
of more cumbersome expressions in Theorem 11.11 Arguably, the 
interesting cases are anyway when the total number of tokens, and 
in particular is polynomial). Complete details in Section HI 
• For the CCC„, after logn rounds the discrepancy is whp at most 

3 (^ — a) logn + loglogn + 0(1). 

Full details in Section [5l 
Let us now turn to the lower bound. Here we consider the all-up-orientation 
of the balancers of a CCC„ meaning that before the a-perturbation, all 
balancers are directed to the wire with a smaller number. 

Theorem 1.2. Consider a CCC„ with the all-up orientation of the balancers 
and assume that the number of tokens at each wire is uniformly distributed 
over {0, 1, ... ,n — 1} (independently at each wire). The discrepancy of the 
a-perturbed network is whp at least 



max 



^ — a) logn — 2 log logn, (1 — o(l)) (loglogn)/2|. 



.2 

Two more points to note regarding the lower bound: 

• For a = our lower bound matches the experimental findings of 



12l |. The authors examined CCC's of size up to 2^*^ where all bal- 
ancers are set in the same direction and the number of tokens at 
each input is a random number between 1 and 100, 000. Their obser- 
vation was that the average discrepancy is close to (logn)/2 (which 
matches our lower bound with a = 0). 
• The input distribution that we use for the lower bound is arguably 
more natural than the tailored and somewhat artificial ones used in 
previous lower bound proofs [l^, 16]. 

Finally, we state a somewhat more technical result that we obtain, which 
lies in the heart of the proof of the lower bound and sheds light on the 
mechanics of the CCC in the average case input. In what follows, for a 
balancer b, we let Odd(b) be an indicator function which is 1 if b had an 
excess token. By Bi we denote the set of balancers that affect output wire 
i (that is, there is a path through consecutive layers from the balancers to 
the output wire i). Bi{l) is the restriction of Bi to balancers in layer L 

Lemma 1.3. Consider a CCC„ network with any fixed orientation of the 
balancers. Assume that the number of tokens at each wire is uniformly dis- 
tributed over {0, 1, . . . , n — 1} (independently at each wire). Then every 
balancer b in layer I, 1 ^ ^ ^ log n, satisfies the following properties: 



'^We point out that in the original statement in Corollary 5], only the number of 
periods is counted. Hence, in their statement the number of rounds is by a factor of d 
smaller. 
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• Pr [Odd(b) = 1] = 1/2 



• moreover, for every wire i, {Odd(b) | b G Bi} is a set of independent 
random variables. 

The proof of the lemma is given in Section [231 Let us remark at this point 
that the lemma holding under such weak conditions is rather surprising. 
First, it is valid regardless of the given orientation. Secondly, the Odd's 
of the balancers that affect the same output wire are independent. While 
this is obvious for balancers that are in the same layer, it seems somewhat 
surprising for balancers in subsequent layers that are connected. 

1.5. Paper's Organization. The remainder of the paper is organized as 
follows. We set out with the proof of Theorem 11.21 in Section [2l preceding 
the proof of Theorem 11.11 in Section [3l The reason is that the lower bound 
is concerned with the CCC„ (a special case of our general model). The 
techniques used in the proof of the lower bound serve as a good introduction 
to the more complicated proof of the upper bound. In Sections H] and [5] we 
show how to derive the special cases of the periodic balancing network and 
the CCC„ from Theorem 11.11 Finally we present experimental results that 
we obtain for the CCC„ in Section [6l 



As we mentioned before, for the special case of the CCC„ we adopt a 
"topographical" view of the network: calling the vertices wires, the time 
steps layers, the left-most side of the network the "input" and the right- 
most the "output" . 

The proof outline is the following. Given an input vector x (uniformly 
distributed over the range {0, . . . ,n — 1}), we shall calculate the expected 
divergence from the average load fi = ||x||i/n. The expectation is taken over 
both the smoothing operation and the input. After establishing the "right" 
order of divergence (in expectation) we shall prove a concentration result. 
One of the main keys to estimating the expectation is Lemma 11.31 saying 
that if the input is uniformly distributed as above, then for every balancer 
b, Pr [Odd(b) = 1] = 1/2 (the probability is taken only over the input). 

Before proceeding with the proof, let us introduce some further notation. 
Let yi be the number of tokens exiting on the top output wire of the network. 
For any balancer b, ^(b) is an indicator random variable which takes the 
value —1/2 if the balancer b was perturbed, and 1/2 otherwise (Recall that 
all balancers are pointing up before the perturbation takes place). 

Using the "standard" backward (recursive) unfolding (see also [l^ . [l^ for 
a concrete derivation for the CCC„) we obtain that, 



2. Proof of the Lower Bound 



log n 



yi=/x+^2-i°s 



n+i 



Odd(b)-*(b). 



i=i 



beBiW 



8 



SMOOTHED ANALYSIS OF BALANCING NETWORKS 



The latter already implies that the discrepancy of the entire network is at 
least 

log n 

yi-^=^2"i°s"+^ ^ Odd(b) • M/(b), 

i=l bsBiOT 

because there is at least one wire whose output has at most fi tokens (a 
further improvement of a factor of 2 will be obtained by considering addi- 
tionally the bottom output wire and proving that on this wire only a small 
number of tokens exit). 

Write Hi — n = Y^^^i Si, defining for each layer 1 ^ £ ^ logn, 
(1) 5^:=2-i°s"+^ Odd(b) • ^'(b). 

2.1. Proof of — a) logn — 2 log logn. We now turn to bounding the 
expected value of Si. Using the following facts: (a) the Odd(b) and ^'(b) are 
independent (b) Lemma 11.31 which gives E[Odd(b)] = 1/2, (c) the simple 
fact that E [^(b)] = i - a and (d) the fact that in layer £ there are 2^°^""^ 
balancers which affect output wire 1 (this is simply by the structure of the 
CCC„), we get 

beBi (£) 

= i(^-«)- 



log n 



^ - a) logn. 



i) log 



This in turn gives that 

E[yi-/x]=E 

.e=i 

Our next goal is to claim that typically the discrepancy behaves like the 
expectation; in other words, a concentration result. Specifically, we apply 
Hoeffdings bound to each layer Si separately. It is applicable as the random 
variables 2"'°^"+^ • Odd(b) • ^'(b) are independent for balancers within the 
same layer (such balancers concern disjoint sets of input wires, and the 
input to the network was chosen independently for each wire). For the 
bound to be useful we need the range of values for the random variables to 
be small. Thus, in the probabilistic argument, we shall be concerned only 
with the first log n — log log n layers (the last log log n layers we shall bound 
deterministically) . We use the following Hoeffding bound: 

Lemma 2.1 (Hoeffdings Bound). Let Zi, Z2, . . . ,Zn be a sequence of inde- 
pendent random variables with Zi G [a-j, bi] for each i. Then for any number 

-2 \ 



Pr [\U=i E ELi Z^]\ ^ ^ 2 • exp 



2e^ 
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For any random variable X, let Range [X] be the difference between the 
maximum and minimum value that X can attain. For a balancer b E 
we plug in, 

Zb = 2-i°g"+^-0dd(b)-^'(b), 

g _ 2(^— log"-+loglog"-)/2 

Range [Zb]' = (2^-i°s")', 
and the sum is over 2^°^""^ balancers in layer L Therefore, 



Pr 



- E [St\\ ^ 2(^-i°s"+'°gi°g")/2 

2 . 2'^~log"+loglogn 



^ 2exp 



2^— logr 



In turn, with probability at least 1 — logn/n (take the union bound over at 
most log n Si terms) : 

logn— loglogn logn— loglogn 

5£ ^ i (i -a) (logn-loglogn)- 2(^-i°g"+^°ei°g")/2_ 

The second term is just a geometric series with quotient -v/T) and therefore 
can be bounded by -; — ^ < 4. 

1-1/V2 

For the last loglogn layers, we have that for every i, \S£\ cannot exceed 
^, and therefore their contribution, in absolute value is at most ^loglogn. 
Wrapping it up, whp 

log n 

yi - ^ = ^ 5£ ^ i (i - a) (logn - loglogn) - 4 - i log logn. 

e=i 

The same calculation implies that the number of tokens at the 6oWom-most 
output wire deviates from fi in the same way (just in the opposite direction), 
so 

Un- fJ-^ - ^ (log " ~ log log n) + 4 + i log log n. 

Hence, the discrepancy is whp lower bounded by (using the union bound 
over the top and bottom wire, and not claiming independence) 

yi-Vn'^ (i - a) log n - 8 - (| - q) log log n 

^ — a) log n — 2 log log n. 

2.2. Proof of (1 — o(1)) loglogn/2. The proof here goes along similar lines 
to Section 12.11 only that now we choose the set of balancers we apply it to 
more carefully. By the structure of the CCC„, the last x layers form the 
parallel cascade of n/2^ independent CCC subnetworks each of which has 2^ 
wires (by independent we mean that the sets of balancers are disjoint). 
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We call a subnetwork good if after an a-perturbation of the all-up initial 
orientation, all the balancers affecting the top (or bottom) output wire were 
not flipped (that is, still point up). 

The first observation that we make is that whp (for a suitable choice of 
X, to be determined shortly) at least one subnetwork is good. Let us prove 
this fact. 

The number of balancers affecting the top (or bottom) wire in one of 
the subnetworks is Yli=i ^^"^ ^ total, there are no more than 2^"*"^ 

affecting both wires. The probability that none of these balancers was 
flipped is (using our assumption a ^ 1/2) (1 — a)^"^^^ ^ 2"^''^ . Choos- 
ing X = log log n — 2, this probability is at least n~^^'^; there are at least 
n/ logn such subnetworks, thus the probability that none is good is at most 



Fix one good subnetwork and let /i' be the average load at the input to 
that subnetwork. Repeating the arguments from Section 12.11 (with a = 

0, logn re-scaled to x = log logn — 2, and now using the second item in 
Lemma 11.31 which guarantees that the probability of Odd(-) = 1 is still 1/2, 
for any orientation of the balancers) gives that in the top output wire of the 
subnetwork there are whp at least /i' -|- (log log n)/4 — ©(log log log n) tokens, 
while on the bottom output wire there are whp at most ^' — (log log n) /4 -|- 
©(log log logn) tokens. Using the union bound, the discrepancy is whp at 
least their difference, that is, at least (loglogn)/2 — ©(log log logn). 

2.3. Proof of Lemma 11.31 The following observation is the key idea in 
proving Lemma II. 3i Recall that Bj stands for the set of balancers that 
affect wire j. For a balancer b in layer i let A(h) describe an assignment of 
Odd(b') values for all balancers b' in preceding layers that affect b (that is, 
there is a path from b' to b through consecutive layers). 

Lemma 2.2. Consider a CCC„ network with any orientation of the bal- 
ancers. Assume that the number of tokens at each wire is uniformly dis- 
tributed over {0,1,..., n — 1}. Consider a balancer b in layer I with 
1 ^ i ^ logn and let xi,X2 denote the two input wires that go into b. Then 
for any assignment ^(b), Xj mod (n/2^~-^) | ^(b) is uniformly distributed 
over {0, 1, ... , (n/2^-1) - 1}, for i = l,2. 

The lemma easily implies Lemma 11.31 Consider a balancer b in layer 

1, with 1 ^ i ^ logn with two inputs xi,X2. Lemma 12.21 implies that 
Xi mod (n/2^~^) is uniformly distributed over {0, 1, . . . , n/2^~^ — 1}, and in 
particular is uniform over that range mod 2. Hence both xi,X2 are odd/even 
with probability 1/2. Further observe that by the structure of the CCC net- 
work, the input wires xi, X2 depend on disjoint sets of wires and balancers. 
By the two latter facts it follows that the sum xi -\- X2 is odd (or even) 
with probability 1/2, or in other words Pr [Odd(b) = 1] = 1/2. The inde- 
pendence part follows from the fact that this is true for every conditioning 
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A{h) on balancers from previous layers that affect b (by Lemma 12.20 and 
the fact that balancers in the same layer that affect the same output wire 
are independent by construction, as those balancers depend on disjoint sets 
of balancers and disjoint sets of input wires. 

Proof of Lemma \2.S\ We prove the lemma by induction on the layer of 
the balancer. The base case is immediate: the input to a balancer in layer 
1 is just the original input, which is by definition distributed uniformly over 
{0, 1, . . . , n/2^~^ — 1} which is simply {0, 1, . . . , n — 1}. Assume the lemma 
is true for all balancers in layer i and consider a balancer in layer i + 1. 
Let xi,X2 be its two input wires. By the structure of the CCC, the value on 
each wire is determined by a disjoint set of balancers and preceding wires, 
therefore we can treat, w.l.o.g., only xi. Let ^i(b) be the part of A{b) 
that affects xi. {Ai{h) would be the set of balancers so that there is a 
path from them to xi. Since the initial load on the input wires is chosen 
independently at every wire, and the sets of balancers affecting xi and X2 
are disjoint, indeed only Ai{b) affects xi, and the same applies for A2{b) 
and X2). Thus our goal is to calculate 



Pr 



xi = /cmod(n/2^) I ^i(b) 



Let b' be the balancer in layer £ whose one outlet is xi, and let x'i,X2 be its 
two inputs. Recall that xi = [{x'i+x'2) /2\ , and a possible +1 addition in case 
this sum is odd and the balancer points in the direction of xi. Furthermore 
it is easy to verify that for every a (assume a is even, if odd write a + 1 
instead) 

a/2 = kmod (n/2^) ^a = 2A;mod (n/2^-^). 
Therefore for the event xi = A; mod (n/2^) to occur, either 

• x'l + x'2 = 2k mod (n/2^~"^) , or 

• x'l +x'2 = 2k — l mod (n/2^~^) (assuming w.l.o.g. that the balancer 
points towards xi, otherwise the sum equals 2k + 1). 

Let us consider the first case. 



Pr 



x'l + x'2 = 2A:mod fn/2^-^) | ^1(6) 



Pr 



n/2*-i-l 

\l x'l = imod (n/2^"^) A x'2 = 2k - imod (n/2^"^) | Ai{h) 



1=0 



Now observe that the values of x'l and x'2 are determined independently, as 
again, by the structure of the CCC, they involve disjoint sets of balancers 
and input wires. Similarly to Ai[b) we can define A'i{b) and (6) which 
correspond to the parts of A affecting x'l and Xg. By the structure of the 
CCC, A'lib) and A'2{h) depend on a disjoint set of input wires (and balancers). 
As the input is chosen independently for every wire, A!2{h) does not affect 
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x'l and similarly A[{b) does not affect Thus the latter reduces to 

n/2^-1-1 

Pr \x[ = imod (n/2^"i) | A[{b) 



i=0 



Pr 



4 = 2A; - imod n/2*-i | ^2(6) 



By the induction hypothesis (applied to the x[ at layer 
£ — 1), for every i, each of Pr [x'^ = i mod (n/2^^-'^) | ^'^(6)] and 
Pr [^2 = 2A; — i mod (n/2^~^) | ^42(6)] is uniformly distributed over the 
range {0, . . . ,n/2^~^ — 1}, and therefore in particular the entire expression 
does not depend on k, or, put differently is the same for every choice of k. 
The same argument holds for the case x'^ + X2 = 2A; — 1 mod (n/2^~^) . This 
completes the proof. □ 



3. Proof of the Upper Bound 

We shall derive our bound by measuring the difference between the num- 
ber of tokens at any vertex and the average load (as we did in the proof of the 



lower bound for the CCC„). Specifically we shall bound maxj \y, 
being the number of tokens at vertex i at time t (we use y 



i 



, (*) 
)i&v for 



the vector of loads at time t) . There are two contributions to the divergence 
from ^ (which we analyze separately): 

• The divergence of the idealized process from /i due to its finiteness. 

• The divergence of the actual process from the idealized process due 
to indivisibility. 

The idea to compare the actual process to an idealized one was suggested in 
17| and was combined with convergence results of Markov chains. Though 
we were inspired by the basic setup from 17|] and the probabilistic analysis 
from [9], our setting differs in a crucial point: when dealing with the case 
< a < 1/2, we get a delicate mixture of the deterministic and the random 
model. For example, the random variables in our analysis are not symmetric 
anymore which leads to additional technicalities (cf. Lemma l3.2p . 

Formally, let be the load vector of the idealized process at time t, 
then by the triangle inequality (1 is the all-one vector) 



Mil 



M _ pit) 



Proposition 3.1. Let G he a balancing network with matchings 
M(i),...,M(^). Then, 

• iie(*2)-/ui||oo ^ A2, 

• whp over the a-perturbation operation, 

||y(*^) - C^*^) Hoc ^ {t2 - ti) + 3 (^ - a) ti + Ai. 
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Theorem 11.11 then follows. The proof of the first part of the proposi- 
tion consists of standard spectral arguments and is given in Section 13.11 for 
completeness. The proof the second part is more involved and is given in 
Section 13.21 



3.1. Proof of Proposition \SA\ Bounding - Letting 

be the initial load vector, it is easily seen that 

^(t2) =^(0)p(l)p(2)...p(t2)^ 

where P^*^ is the matrix corresponding to matching M^*^ (as defined in 
Section [1.3p . For simplicity let us abbreviate 

p(i+l) p(i+2) . . . p(j)^ 

Since p[-^'*2] is real valued and symmetric (as each of the P*^*)'s is), it has n 
real-valued eigenvalues Ai ^ A2 ^ • • • ^ A„ whose corresponding eigenvec- 
tors vi, . . . ,Vn form an orthogonal basis of M". Next we observe that 

_ ^ 1 = ^(0) p[i,t2] _ ^ 1 p[i,t2] ^ (^(0) _ ^^-^ pUM 

since 1 is an eigenvector of pt^'*^] corresponding to Ai = 1. Furthermore 
S,^^^ ■ 1 is just the total (initial) number of tokens, and therefore by definition 
we get {^^^^^ — /il) -1 = 0. Finally, let us project S,^^^ — onto vi, . . . ,Vn, 
that is, write — fil = Y27=2'^i'^i ("^i = as we said). For our goal to 
bound 11^^*2-* — /^l||oo, it suffices to bound ||^*-*^^ — ^1|| (recall that ||.|| refers 
to the ^2-iiorm) as for every vector z, \\z\\oo ^ ||^;||. By the above. 



|e(*2)-^l|| = ||(e{0)-^l)pli' 



t2]\ 



i=2 



Vi 



i=2 



Recall that A(p[^'*^l) denotes the second largest eigenvalue of pl^'*^] in ab- 
solute value. By the definition of the ^2-iiorm, and using the fact that the 
v[s form an orthogonal basis, the latter equals 



1/2 



a(p^*^i)- E 

\i=2 

=A(P^*^1).||^W-^1 

^A(p[i'*2i)E:^. 



1/2 
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3.2. Proof of Proposition [3. It Bounding lly^*^^ — C^*^^l|oo- The proof 
of this part resembles m nature the proof of Theorem II. 2[ Assuming an 
ordering of G's vertices, for a balancer b in round t, h = {u,v), u < v, we 
set ^u^v = 1 if the initial direction (before the perturbation) is u — )• u and 
— 1 otherwise (in the lower bound we considered the all-up orientation thus 
we had no use of these variables). As in Section [2l for a balancer b = {u,v) 
in round t, the random variable is —1/2 if the balancer is perturbed 
and 1/2 otherwise. Using these notations we define a rounding vector p^^\ 
which accounts for the rounding errors in step t. Formally, 

Odd(2/i*-') 

if u and v are matched in M*^*-* and u < v, 



yv ) ^ u,v ^u,v 



-Odd(y. 



(t-i) 







if u and v are matched in M*^*-* and u > v, 
if u is unmatched. 



Now we can write the actual process as follows: 



(2) yW =y(t-l)pW+pW. 

Let M^^^^ be the set of balancers at time t with no excess token, and Mq^^ 
the ones with. We can rewrite /j^*^ as follows: 

Unfolding equation ([2]) , yields then 

t 

y(t) ^ y(0)p[l,i] + ^pWp[i+l'*]^ 
1 = 1 

where pl^'^l = I. Observe that y(o)p[i'*l is just C^*) (as = y(°)), and 
therefore 



yW _ ^(t) = J2t^_^ p(Op[i+l,t] 



In turn. 



Our next task is to bound equation ([3]) to receive the desired term from 
Proposition 13.11 We do that similar in spirit to the way we went around in 
Section 12.11 We break this sum into its first ti summands (whose expected 
sum we calculate and to which we apply a large-deviation-bound). The 
remaining (t — ti) terms are bounded deterministically. 
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One major difficulty in the general setting is that Lemma [1.31 (which was 
crucial in the proof of Theorem II. 2p does not hold in general as its proof 
makes substantial use of the special structure of the CCC. 

Equation [3] with t = t2 yields 

With Cu^v denoting the row- vector with +1 at u-th column and —1 at v- 
th column and zeros elsewhere, we can rewrite and split this equation as 
follows: 



Clearly, 

= Ttu,r i^^^.M^, *a e.,) . (P[^-^^.*^] e.) . 

Observe that V, , ,ti) "^u\^u\e.uv is a vector all of whose entries are 

bounded by 1 in absolute value. Since p[*+i'*2] ig a stochastic matrix, the 
sum of all entries of the w-i\i column of p[*+i'*2]^ which is |p[*+i'*2] e^|i, is 
exactly one, hence 

E -Lt.+i (E(„,„),m(:) ■ I ^ - h). 

It remains to bound 

Because VFodd is not necessarily a sum of independent random variables, it 
will be more convenient to work with the following quantity (which is a sum 
of independent random variables, as it assumes that every balancer gets an 
excess token), 

So our strategy is first to bound the deviation of W from its mean by Ho- 
effdings Bound and then apply the following lemma (whose proof is in Sec- 
tion [33]), which justifies using W instead of Wodd- 



Lemma 3.2. 


Fix ^ 




weights Wu}v £ 


R, let 






Wodd 


V^t2 




WEyen 


V^t2 




w 


y^t2 



,T,(*) <T)(*) „,,(«) 

,T,(«) „,,(*) 

=<i Z^(u,-u)eM(') ^M,-" ■ 
Then for any (5 > 0, 

Pr [l^^oddl > 6 + 2max{| E [W+] |, | E [W^] |}] ^ 16 Pr [|^| ^ 5] , 
where 

tn(') >0 

In order to apply Lemma 13.21 we first need to bound the expectation: 



E 



„ p[i+l,t2]p 



^ max max 



E 



As we explained before, the last term is at most ti, and for any i £ 
and {u,v) G M», E [^-Jl ^ (1/2 - q). Thus, 

|E[Ty]| ^ ti (1/2 - a). 

Similarly, |E [W+]\ ^ ti (1/2 - q) and |E [W'W ^ ti (1/2 - a). 

To apply Hoeffdings bound on W , we bound the sum of the squared ranges 
of the involved random variables as follows: 

T!U E(„,.)6M« Range [^>% <D« • e„,,p[*+i'*2]e^] ' 

Now by Hoeffdings Bound, 
Pr [\W\ ^ I E [T^] I + e] ^ Pr - E [VF] I ^ e] 
^ 2 exp (-2ey Eti E(„,.)eA./W Range [^£1 (e„,„p[^+i'*2]e^ 
^ 2 exp (-2eV7) • 
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Choosing e = -y/7 logn' we get Pr ^ | E [1^] | + ^/t logn'] ^ 2n ^. 
Hence by Lemma 13.21 we obtain with = | E \W] \ + logn' that 



Pr 



iW^Oddl ^ |E[l^]|+2max{|E[t^+] |,|E[VF"] |} + V7logn' 



^ 16 Pr 

^ 32 n"^ 



\W\ ^ |E[VF] I + V7log 



n 



As ii (1/2 — a) is an upper bound on each of | E [T^] |, | E [T^+] | and 
I \W~] I) this readily implies 



Pr 



iH^Oddl ^ 3 (1/2 - a)ti + v/7log 



n 



^ 32n-^ 



Finally, taking the union bound, we conclude by equation ([3]) that for all 
vertices w 



Pr 



A 



^3(l/2-a)ti+7Vlogn') 



> 1 - 32n 



-1 



3.3. Proof of Lemma 13.21 Before we begin the proof of Lemma 13.21 we 
require the following two technical lemmas. 

Lemma 3.3. For ^ a ^ 1/2, arbitrary Wi G M, and independent 
random variables Xi that are —1 with probability a and 1 otherwise, let 
X := Yl'^^i WiXi. Then, 

• // all Wi ^ 0, then Pr [X < 0] ^ 1/2, 

and for any number 5^0, Pr [X > 5] ^ Pr [X < —5]. 

• // all Wi ^ 0, then Pr [X > 0] ^ 1/2, 

and for any number 5^0, Pr [X < —6] ^ Pr [X > 6]. 

Proof. Note that it suffices to prove the statement with all Wi ^ 0, as the 
case with Wi ^ follows from the first by considering X' := Y17=i '^i-^i 
with w[ := —Wi. Let Y := Y17=i ^i^i where each Yi is an independent and 
uniform random variable taking values in { — 1, +1} (corresponding to X with 
a = 1/2). As y is a sum of symmetrical distributed random variables, we 
have Pr [y < 0] ^ 1/2. Since for any a ^ 1/2, X stochastically dominates 
y, we obtain Pr [X < 0] ^ Pr [Y < 0] < 1/2, and the first claim of the 
lemma follows. 

The second claim is proven similarly by observing that 

Pr [X > (5] ^ Pr [Y > 5]= Pr [Y < -5] ^ Pr [X < -5] . □ 



The following lemma bounds the probability that a sum X := XliLi 
deviates by more than a factor two from its expectation. 



WiXi 



Lemma 3.4. For ^ a ^ 1/2, arbitrary Wi G Q, and Xi independent 
random variables that are —1 with probability a and 1 otherwise, let X : = 
Y17=i'^i^i- Then, 

(i) // all Wi ^ 0, then Pr [X > 2E [X]] 7/8. 
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(ii) // all Wi ^ 0, then Pr [X < 2E[X]] ^ 7/8. 

Proof. As before, it suffices to prove tlie first statement. Moreover, we may 
focus on tlie case E [X] / 0, as otherwise X is symmetrically distributed 
around 0. Finally, we also assume the weights to be integral, that is, Wi G 
Z^o- Rational weights Wi G Q^o can be easily reduced to integral weights by 
multiplying all weights with their least common multiple of the denominators 
and applying the bound for the integral case. 

For the sake of contradiction, suppose that Pr [X > 2E [X]] > 7/8. As 
^ a ^ 1/2, we have E [X] = Y,1^i Wi{'^ - 2a) ^ and hence Pr [X ^ 0] ^ 
Pr [X ^ 2E [X]] < 1/8. Let A: > 2E [X] be such that Pr [X k] ^ 1/8, 
and Pr [X > k] < 1/8. 

As we assumed the weights Wi to be integral, we can use the following 
two well-known counting tricks: 



and similarly, J^x^-fc ~^ [X = x] = Ylx^i ^ — max(A:, x)] . 

We use both to obtain 



X 




y^l a;^max(fc,j/) 





= Pr [X ^ A:] + ^ Pr [X ^ x] 



1^3;<fc x^k 



> Y 0] + Pr [X^k]- 1/8) 



l^a:<A: 



+ ^Pr [X-^x]. 



By Lemma 13.31 we now get 




+ j;Pr[X^ 



x] 



k (Pr [X ^ k] 



1/8) +^Pr [X ^ -x] 



k (Pr [X ^ k] 



1/8) + Y -^Pr [X = x] 



x^-l 
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Plugging this in the definition of E [X] , we get 
E[X] = ^xPr[X = x] 

X 

= ^ X Pr [X = x] + ^ xY>y[X = x] + ^xVt[X = x] 
^k(Pr[X^k]-l/8)+ xPr[X = x]. 

0^x<k 

Using above assumptions on k and Pr [X ^ 2 E [X]] > 7/8 we now arrive at 
the desired contradiction, 

E[X] > 2E[X] (Pr[X ^ A;] - 1/8) + ^ xPr[X = x] 

2E[X]!^x<k 

^ 2E[X] {Pr[X ^k] - l/8 + Pr[2E[X] X < k]) 
^ 2E [X] (Pr [X ^ 2E [X]] - 1/8) 

^3/2E[X]. □ 

We are now ready to prove Lemma [3.2[ In the following, we will use subsums 
of W denoted as W^^^^ , W^^^^ , Wq^^ , Wq^^ which are defined by combining 
the previous definitions in a natural way, e. g., 

''''Odd •- Z^i=ti („^„)gM^;>, '^u.v'i^u.vWu.v 

Let M[*1'*2] = y*^^^ m(*) be the set of all matching ed ges in the given time 
span. Let M^*^'*"' be the set of all odd ones. Moreover, let us simply write 
M for a particular assignment of Odd(.) for all balancers in mI*!'*^]. We now 
begin by bounding VFodd in terms of W . By the definition of conditional 
probability and the law of total probabilities, it follows for arbitrary 81,82 € 



Pr [VF < (52 I VFodd ^ 5i] = - 



Pr [VFodd > 81] 
Pr [W - Wo6d <82- 81 A l^odd > 81] 



Pr 



Pr [WoM ^ 81 
Pr 



V^Even <82-8l^ V^Odd > 81 \ M^^'f/ = M 



^ Pr[M^Odd5s5i] 

Note that for fixed M, the ranges of Wodd and are determined and 

therefore the probability spaces are disjoint. This implies that Wodd and 
M^Even are independent conditioned on Mq^^^^ = M. Using this observation. 
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we get 



Pr [W <52\ ^ h\ 



(4) 



M, 



Odd 



M 



Pr 



<^2-b, I Ml^i^ 



M 



Pr 



7W 

Pr [WoM ^ h\ . 



By plugging (^2 := <5 + 2 E [VF ] and b\ := (5 in (*) we obtain 



Pr 



P^Even < 2 E [T^-] I Ml'i^^ = M 



= 1 - Pr 
^ 1 - Pr 



Even > 2 E [W~] I Mq^^ 



[tl,t2] 

Od 

M 



M 



Pr 



Even 



^ 2 E [W-] I M, 



Odd 



M 



Observing that E [^u, 
E [W-] ^ E 



E 



•^•^Even I ^'^Odd 



u,v 

M 



for {u,v) G A^Even' 



and hence by Lemmas 13.31 and [37 



Pr 



W^Even < 2 E [W-] I m[)*^'*^] = 

< 1 - Pr 



•^•^Even ^ ^ I ^'^Odd 



M 



Pr 



W^E^ven 2 E 



Even 



[tiM 

Odd 



[tl,t2] 



Odd 



M 



1 1 

2 ■ 8 



15 
16- 



(5) ^1 
Plugging this into equation 



yields 



Pr[W <6 + 2E [W-] I Wodd > S] 



15 
16 



E 

M 



Pr 



M, 



Odd 



M 



Pr 



T^Odd ^ Si I Mi*^'*^l = 



Pr [VFodd > Si] 



15 
16 



and hence 



Pr [Ty ^ 5 + 2E [W-]] 

^ Pr [Ty ^ (5 + 2E [W-] A Wodd ^ S] 
= Pr [T^odd ^ <5] • Pr [VF ^ 5 + 2E [W-] \ Wq^ > S] 
(6) Pr [Vf^odd ^ S] . 
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It remains to lower bound the deviation of VFodd in terms of in a similar 
fashion. As before, we derive 



Pr [W>52 \ P^odd ^ h] 
(7) ^ E P 



m£'*^1 = M 



Pr 



M^Even > 52 - 5l I M^'f^"^ = M 



M 



Pr 



W^Odd ^ '^1 I M^*^'*^] = / Pr [WoAA ^ h] . 



We now choose (52 := — (5 + 2 E \W^] , and 5i := —5. As before in equa- 
tion (0), we can now use Lemmas 13.31 and 13.41 to get for (*) that 



Pr 



M 



^ 16- 



W^Even > 2 E [W+] I Ml^^ 

Plugging this into equation ([7]) yields 

Pr [ly > -(5 + 2E | Wodd ^ -5] ^ 15/16 

and hence 

Pr [W ^ -(5 + 2E [Ty+]] 

^ Pr [VF ^ -5 + 2 E [Ty+] A Wodd ^ 

= Pr [VFodd ^ -5] • Pr [H^ ^ -5 + 2 E [W+] \ Wodd ^ -5] 
(8) ^ i • Pr [l^odd ^ -<5] . 

Combining equation ^ and equation ([8|), we have shown for any 5^0, 
^v[\Woaa\^5] 

= Pr [H^odd ^ 5] + Pr [l^odd ^ 

< 16 Pr [W" ^ 5 + 2E [W-]] + 16 Pr [W ^ -5 + 2E [W+]] 
^ 16 Pr [\W\ ^ 5 - 2max{| E |, | E [W] |}] . 

Adding 2max{| E \W~^] |, | E \W~] |} to 5 gives the assertion of the lemma. 



4. Deriving the Upper Bound for the Periodic Case 

Consider the network after "^^^^^^q^ (periodic) repetitions of the balancing 
network (each repetition consists of d rounds, so the network consists of 
a total of T = d • matchings). Q is the so-called round matrix 

corresponding to the application of d consecutive matchings (recall that 
we use the following abbreviations p[*i'*2] = ]^*^^^ P^*)). This is indeed a 
special case of our general scheme where the matrices P^^^ , P^^^ , • • • , P^^^ are 
applied, but now Q = pl^-'^l = ^j^^ gQ Recall the notation that 

e„ „ denotes the row- vector with -|-1 at n-th column and —1 at t'-th column. 



In Theorem 11.11 we plug t2 = T 



2d\og{Kn) 



i-A(Q) - 1 i_A(Q) • ilence. 
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Ai = max 0ogn E*Li E(«,.)eMW {eu,vP^'+'^''^e^f . 
Let v^\v2 \ ■ ■ ■ ,Vn^ be the eigenvectors of p[*+i'*2] forming an orthogonal 

(i) (i) (i) 

basis of M" and let A\ , i . . . , An be the corresponding eigenvalues. Since 
p[«+i,t2] jg stochastic matrix, v^^ = 1. Write = J2^=i '^f'^ '^f^ ■ Using 
this, we rewrite Ai as follows (for short we drop the "max^" part as our 
final result will not depend on ti;), 

= ^logn J2lLi E(«,^)eM(») (en,i> E]=i 4^>'?'^f) ■ 

Observe that eu,v is orthogonal to Therefore c^*^ = for every i which 
gives 

Ai = ^logn E*LiE(„,.)eM« (e«,.E-=2f A«.f )''. 

Define a vector z^*) := ^^^=2 ^^^'^i*^^]*^' '^^^ latter is then 
Ai = 0ogn EiLi EK„)eA/W {eu,vz^''^y 
= ^logn 'E{u,v)eAm (^"^ - ^^*^) 

Since M^^) is a matching, for each i each vertex is counted only at most 
once, thus 

(9) Ai ^ 20ognEtill^^'^f '• 

By standard calculation (cf. Section [3. ip . we have 

Plugging this into Q and using the facts that for any two stochastic matrices 
A and B, A(AB) ^ A(A) and for any integer k, A(A'^) = A(A)'^, 

Ai ^20ognE-Li^(P''+''*'^)'' 

^ 2^1ogn E*LiA(QL(*2-(i+i))/'iJ)2'. 

Regrouping the matrices according to the periods the latter reduces to 
Ai ^ ^logn E*=i''rfA(Q)2fe/'^-(^+i))'. 
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A(Q)2P 



Plugging in our choices for ti and t2 we end up with 



I AfQ)-i+4i°gios"/(i-MQ)) ' 

1 



exp(-3 log logn) 
^ 2 W d log n ■ 



^2-^Jd ^ 



I -my 



lognV 1-A(Q)2 
2^/d' 1 



logn l-A(Q)" 
With the same arguments, we get 

A2 ^ A(p[i'*2l) ^ y/^K/{Kn? 



Plugging all this into Theorem 11.11 shows that at step T = the 
discrepancy is at most 

2d log logn 3(1/2 - a) • 2d log(/s:n) 2y/d! 1 y/KK 
l-A(Q) ^ 1 - A(Q) ^ bi^ ■ 1 - A(Q) ^ {KnY 

^n. , d'^ogiKn) (I \ dloglogn\ 



5. Deriving the Upper Bound for CCC^ 

In Theorem 11.11 we choose ^2 = logn and ti = logn — log logn. For 
matrix multiplication we use the following abbreviated form: pl*'-?] : = 
p(i)p(i+i) . . . p(i). First we observe that 

• Piiji;^^ = 2'^/n if wires u and v are at distance ^ log n — k and differ 
only in the last log n — k bits (in their binary representation) , 

• otherwise, Pu,J;*^^ = 0. 
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0.1 0.2 0.3 0.4 0.5 



parameter a 



Figure 2. Discrepancy for various a-values of CCC230 with random uniformly dis- 
tributed input from {0, 1,2,..., 2^^ — 1}- a = corresponds to the adversarial 
model while a = 1/2 is the completely random model. The dotted line describes 
the experimental results, the broken lines are our theoretical lower and upper 
bounds. 



In particular this shows that pl^'*^] ig equal to the all-(i) matrix and thus 
A (p[^'*2]^ = (that is, A2 = 0). Moreover, for any fixed wire tf, 

, /i v-^logn— loglogn n 1 2*+^\'^ 

A /i v-^logn— loglogn 2' ' 

= 4ylogn l^-J^ n 

I 2^ 
^ 4 Wlogn • 

V log n 

= 0(1). 



6. Experimental Result 

We examined experimentally how well a CCC balances a random input. 
We implemented a CCC230 consisting of roughly one billion wires and thirty 
billion balancers. The input was chosen independently uniformly at random 
from {0, 1, 2, ... , 2^'^ — !}• For initial directions of the balancers all up and 
different values a between and 1/2 we measured the resulted discrepancy. 

Figure [2] presents the average over 100 runs, together with theoretical 
lower and upper bounds. As the input is random, the so-far presented 
bounds can be tightened. Following the same lines of proof, one can easily 
show the following slightly better bounds on the expected discrepancy A in 
the random-input case: 
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• A ^ — a) • (logn — [log log n]) + [log log n] +4, 

• A ^ max{(l/2 - a)logn, 1/2 (1 - i) ([loglognj - 1)}. 

As these bounds are only used for visualization in Figure [2] and the proofs 
are very similar to the above, they are omitted. 
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